Real Time Image Processing Based on Audio Content

This paper explores the connection between audio and visual media by using audio features to control image processing in real time. Anyone involved in the audiovisual industry is aware of the important connection between what we see and what we hear in media. Our goal was to explore ways of mapping audio content to the processing of a static image to cultivate strong connections between the audio being played and the processed image that we observe. Several image processing techniques were explored including color tinting, color contrast, and blurring using a Gaussian filter. The real time processing was implemented in MATLAB using system objects from the DSP and Computer Vision toolboxes. Our implementation was tested using several pieces of classical music, and we believe that some level of success was achieved with much room for improvement and further exploration.